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ABSTRACT 


The Bootstrap method is a nonparametric statistical technique for estimating the sam- 
pling distribution of estimators of unknown parameters. While the asymptotic theory for 
bootstrap 1s well established, this thesis investigates the behavior of the bootstrap for 
small sample sizes. For the exponential distribution and for normal linear regression the 
bootstrap estimates of *he parameters and their variances are compared with the the- 
oretical sampling distributions. The small sample properties of bootstrap confidence in- 
tervals using the percenule method and the bias-corrected percentile method are also 


investigated. 


THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may not 
have been exercised for all cases of interest. While every effort has been made, within 
the time available, to ensure that the programs are free of computational and logic er- 
rors, they cannot be considered validated. Any application of these programs without 


additional verification is at the risk of the user. 
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I. INTRODUCTION 


The Bootstrap method, a statistical technique for estimating the sampling distrib- 
utions of estimators of unknown parameters, was introduced by Efron [Ref. 1] in the 
mid 1970s. This computer intensive method 1s nonparametric in nature and relies on re- 
peated resampling (bootstrapping) from the observed values of a random sample. 

sappe AE are die observed values of a random sample of size n, AL, 
٦۰۰۰۹ ۰٣٦٦٦۱۱ distribution /,(x;0). Let pem h(X,, X, X, ..., X,) be an estimator 
for the unknown parameter 0 . The sampling distribution of ፀ completely describes the 
properties of the estimator and its knowledge would be useful for investigative purposes. 
However in many situations the analvtical derivation of this distribution may be quite 
demanding. An alternative approach is to estimate the sampling distribution using 
ትን ከክ ን 111510 Iv bootstrap samples of size x", , x*5 , x*4 ,.«., x*, for 
ከመ ከን  ከበነኔሜ generated bv repeated uniform sampling with replacement from the 
ML ٦۷۹٣ 0%, xh. x ۱1۹۶ 
for each of the N bootstrap samples. The empirical distribution of the 0%, OS 
3, …, N18 taken as the estimate of the sampling distribution of. 

Efron [Ref. 1] showed, that the bootstrap estimator is consistent and Beran et al. 
[Refs. 2, 3] proved that under fairly general regularity conditions the bootstrap distrib- 
uton converges to the true sampling distribution as n > œo and NX > oo. It has also 
been demonstrated that bootstrap methods perform better than some of the other re- 
sampling techniques such as Hartigan’s subsample method [Ref. 4] and the Tuker- 
Quenouille Jackknife [Ref. 1]. 

Although the asvmptotic behavior of the bootstrap has been well established by 
theoretical research, there are still some problems dealing with the small sample prop- 
erties of the methods, which are open for further investigation. One of these problems 
is the question of how the original sample size n and the number of bootstrap repli- 
cations N affect the “closeness” of the bootstrap distribution to the exact sampling dis- 
tribution. Another one deals with the applicability of bootstrap-based percentiles as a 
basis for estimating confidence intervals for parameters. Information about these issues 
will be useful to a practitioner in the decision of how to emplov his resources. 

The aim of this thesis is to address the two problems stated above. The approach 


which is taken is to consider probability distributions and their parameters, for which the 


exact sampling distributions of the estimators can be derived theoretically. The results 
of simulations of the bootstrap method will be compared with the theoretical results in 
order to analvze the impact of the sample size n and the number of bootstrap repli- 
cations N in the contexi of relativelv small samples. 

Chapter II provides an overview of some bootstrap methods and their properties. 
In Chapter III the bootstrap method is applied to the maximum likelihood estimator 
of the scale parameter of the exponential distribution. The estimation of the parameters 
in normal linear regression is studied in Chapter IV. In Chapter V the conclusions are 


presented. 


Il. BOOTSTRAP METHODS - AN OVERVIEW 


A. THE BASIC BOOTSTRAP METHOD 

The bootstrap method is a resampling technique for estimating the sampling dis- 
tribution of an unknown parameter of a probability distribution. Let X 2 (X, Xj, X,, 
… A, ) be a sample of size n from a distribution with probability density function 
EO) and distribution function F,(x:0). Let 0 h(X) be an estimator for the parame- 
ter 0 . The distribution of ô g(6; 0) 1s called the sampling distribution of 0 . In many 
problems it may be quite difficult to derive the sampling distribution analytically. But 
since computer resources are nowadays inexpensive and easily available, methods like 
bootstrap [Ref. 1], which will be described below, can be used to estimate the distrib- 
ution of 6. 

SE dee ብ A ) are the observed values of the random sample. A 
5۰۰٠٠٠٠٠٦٢٢٢٣٣٢٣ ln x, x, } (~ indicates bootstrapping) of size n is 
obtained by randomly drawing with replacement from the original sample x. Another 
way of describing this resampling procedure is: The empirical distribution function F, 
which is discrete, is constructed by assigning a probability mass 1. n to each of the ori- 
ginal samples x, and then drawing n random samples from F. Although it 1s possible to 
imagine, as Bickel and Freedman [Ref. 3] mention, bootstrap samples of an arbitrary 
size m, mathematical theory [Ref. 3] indicates that the use of the same size n as in the 
original sample is preferable. 

Before continuing the description of the bootstrap method it seems appropriate to 
summarize some properties of anv bootstrap sample. Each element in a bootstrap 
sample is drawn independently from the original sample. So conditional on the original 
sample the probability that the jth element in a bootstrap sample is any one of the oril- 
ginal sample values is the same: 


P(X*, — xix) ብ. Ho A n (2.1) 


n 
The expectation, conditional on X, of X* is 


NS j = 1,2,3,.., n, (22) 


where X is the sample mean Yx/n. Then for example the mean of the bootstrap sample 


$ x*/n has the conditional expectation 





(2.3) ا رو اوہ 


and the unconditional expectation 





ELE EA 3 (2.4) 


The variance of the mean of the bootstrap sample 1s 


n 
Var[X*|x] = dd ر‎ (2.5) 
A SS 


which for n > oo converges to Var[X]. 

The process of obtaining one bootstrap sample set and computing the estimator for 
this sample 1s called a bootstrap replication. For the bootstrap method N bootstrap 
replications are performed, where N varies throughout the literature between 100 and 
2000. This means that N bootstrap samples x*, for) = 1, 2, 3,..., N are obtained and 
for each sample the estimator 0% = h(X*) is computed. The bootstrap distribution, the 
empirical distribution of the 0%, is then an estimate of the sampling distribution of 0. 


The bootstrap estimate lor 0 is defined by 


N 
oe ፌ LY he 
pe = 0 (2.6) 
j=] 
and 
1 : 
Ce ጅር ም - x Eu 
5* >= Haie 6°) (2.7) 


is the bootstrap estimate of a» , the standard deviation of 6. 
Efron [Ref. 1] and Bickel and Freedman [Ref. 3] have shown, that under fairly 
general regularity conditions, as n > oo the bootstrap estimate and its standard devi- 


ation converge to their actual values. 


B. VARIATIONS OF THE BOOTSTRAP METHOD 
This section briefly describes some variations of the bootstrap method to demon- 
strate the variety of options available to the practitioner. These methods however will 


not be the subject of investigations in this thesis. 


l. Parametric Variations of Bootstrap 

To improve the bootstrap method in those cases, where additional information 
about the underlying distribution is available, Efron proposed [Ref. 4] the Smoothed 
Bootstrap. The major difference from the basic bootstrap is, that the bootstrap samples 
are now obtained by sampling from a continuous empirical distribution F. This distrib- 
ution F is constructed by interpolating between the steps of the discrete empirical dis- 
tribution F using an appropriate smoothing function. Efron points out that the choice 
of the function is not arbitrary. In order to gain improvement of the results, compared 
to the basic bootstrap, the selection of the function type has to be compatible with the 
distribution under investigation. So this variation of the method is no longer 
nonparametric in an absolute sense. 

If the exact distribution of the X, is known except for the values of the parame- 
ters, this distribution can be used to perform the smoothing; Efron [Ref. 4] calls this 


method the Parametric Bootstrap. 


2. The Balanced Bootstrap 
Davison, Hinkles and Schechtman (Ref. 5] introduced the Balanced Bootstrap 
to eliminate the iinear component of the bias of bootstrap estimators. Their method 
obtains the N bootstrap samples by first catenating the vector of n original samples N 
times, randomly permutating the resulting vector and then taking N successive vectors 
of size n, ensuring that each x, occurs exactly N times in the total N bootstrap samples. 


It is easily seen, that when an estimator h(X) for @ is linear and symmetric in X, then 


N 
Y 2 Je) E (2.8) 


C. CONFIDENCE INTERVALS 
One of the applications of the sampling distribution 1s to approximate confidence 
intervals for a parameter. The following sections discuss two bootstrap-based methods 


for this purpose. 


1. The Percentile Method 
The percentile method is appealingly straightforward and provides, Efron 
(Ref. 4], good results. It is based on the definition of the empirical cumulative distrib- 


ution function G* of the estimator, 


Es &(0*, « x) 
(7 IU ME — (2.9) 


The pth percentile then can be approximated by ፀ*, defined by 
P* « 0*) « p. (2.10) 


Efron [Ref. 4] proposes the use of (6*,, 0*, as an approximate 100(1 - 2a )% confi- 


dence interval for 0 . 


2. The Bias-Corrected Percentile Method 
The bias-corrected percentile method covers those cases, where the empirical 


bootstrap distribution is not median-unbiased, 1. e., 
P(0* « 0) # 05. (2.11) 


The percentile method may produce inaccurate percentile estimates in this case. To 
compensate for these inaccuracies, Efron [Ref. 4] introduces the Bias-Corrected 
Percentile Method. This method relies, as Schenker [Ref. 6] points out, on an assump- 
tion, which in general is at best approximately valid. The assumption is, that there exists 


a function g such that 


ደ[#) -- ፪[9) — Nn, 1?) (2.12) 
and 
g(0%)— g(0) = N(n, 1?) (2.13) 


with y and 7 being real variables but constant for a specific case. Let 


2, = 07 '[G*(0)] (2.14) 
and 
2, = ወ”'([1--2] (2.15) 


where D denotes the cumulative distribution function of the standard normal distrib- 
ution and @ is the value of the estimator for the original sample. Then the approximate 


1 - 2x confidence interval is given by 
ሀከ... PP ID (2.16) 
It is easily seen, that for median unbiased sampling distributions, 1. e., if 


)2.17( ۴ و ئ 


z, * O and the bias-corrected percentile method is identical with the percentile method. 
Schenker's intention [Ref. 6] is to demonstrate some deficiencies of bootstrap-based 
confidence intervals for small sample sizes. Nevertheless, he does provide results which 
seem to indicate, that the bias-corrected percentile method 1s an improvement over the 
percentile method. 

For the cases, where the underlying assumptions for the bias-corrected 
percentile method do not hold, Efron and Tibshirani [Ref. 7] proposed another method 


called the BC, method. This thesis is concerned with the first two methods only. 


II. THE EXPONENTIAL DISTRIBUTION 


In this chapter, the performance of the Bootstrap method 1s compared to the the- 
oretical results in the case where the underlying distribution is the exponential distrib- 


ution. 


A. THEORETICAL RESULTS 
Let X,, A), A3, … , X, bei. i. d. random samples from the exponential distribution 


Exp [ 4] with probability density function 


—AX 
fide) = | EE (3.18) 
0 otherwise 
The Maximum Likelihood Estimator (MLE) for the scale parameter 4 is 
ja N (3.19) 





Using the fact, that the sum of n i.d. exponential random variables is distributed as 


Gamma [ / ,n ], the probability density function of the random variable W, defined by 


We گے‎ (3.20) 
>»: 
can be shown to be 
EI gar w>0 
fiw) = ከ ie نت‎ (3.21) 
0 otherwise . 


This is the exact sampling distribution of the maximum likelihood estimator for 4 . 
Figure 1 shows the graph of the sampling distribution for 4 = 1 and sample sizes 
n = 10, 30 and 50. 


Computations of the moments yield 


2 
ሆን 
Z 
LU 
iem 
> 
= 
< 
CD 
ርጋ 
2 
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Figure |. Probability Density Function of the Sampling Distribution: 


sample size n = 10, 30, 50. 


ERA == TA 


which shows that the MLE is asymptotically unbiased, and 


(3.22) 


2 
Vari h] = —— 7’. (3.23) 
F (n — Dn — 2) 


For this distribution exact probabilities can be computed using the following identity, 


VN SN MEE (3.24) 
where IJ, denotes the Incomplete Gamma function. Table 1 shows the true values for 


some percentiles of the distribution of W for A = 1. 


Table 1. PERCENTILES OF THE SAMPLING 
DISTRIBUTION: 4 - 1, sample size n. 


aL Le Ln ١ 58۹ 
EE 
0439 















B. THE SIMULATIONS 


]. Point Estimation 

The purpose of this simulation is to investigate the performance of bootstrap 
point estimates. Cortes-Colon (Ref. 8] explored this subject for the sample mean of ex- 
ponential variates, using the mean squared error as the criterion for his evaluation. This 
paper in contrast approaches the problem by looking at the bias and the variance sepa- 
rately in order to isolate effects. 

The simulations in this section were conducted in SIMTBED [Ref. 9], a simu- 
lation software package for the IBM Personal Computer and compatibles, which uses a 
multiplicative congruential generator with multiplier 16807 and modulus 23! — 1 for the 
uniform and an acceptance-rejection scheme for the gamma variates. For the exper- 
iments in this section, ten super-replications were performed with differing numbers of 


trials for each original sample size. The original sample sizes used were n = 10, 20, 30, 


10 


40 and 50 with respective numbers of replications for one super-replication M = 480, 
240, 180, 120 and 96. With 10 super-replications, this sums up to a total of 4800, 2400, 
1800, 1200 and 960 trials for each n and for each of the bootstrap replications. For 
validation purposes, similar simulations were performed on the author’s personal com- 
puter using the APL language and also on the NPS mainframe using independent 
FORTRAN 77 programs. The results were similar to those obtained from SIMTBED. 


a. Bias of the Bootstrap Estimate 

In the first part of the simulation experiment, the quantity of interest is the 
bias B, the difference between the bootstrap estimate for the scale parameter / and its 
true value (4 = 1). The bootstrap estimate À* was obtained according to equation 2.6 
and the bias B was computed as B = À* — 1 for each combination of n and N. 
Figure 2, created with GRAFSTAT [Ref. 10], shows the average values for B as a 
function of the number of bootstrap replications N, for various values of n. Table 2 lists 
the lengths of the central 90% confidence intervals for B, which are based on the 
super-replications. 

The graph of the average values of B shows, that the number of bootstrap 
replications N has on the average almost no effect on the “closeness” of the bootstrap 
estimate to the actual value. Linear regression performed on the averages versus N re- 
sulted in slope parameters of the order of 10-3 and less. The bias is significantly affected 
by the sample size n. The reason for this behavior is the fact that the estimator is biased 
and that the bias decreases with increasing sample size. The average bias for each value 
of n is significantly larger than the amount expected from equation 3.22, which for this 
case would be l(n-1). The observed average bias is approximately twice the expected 
value which seems to indicate that the bootstrap method introduces additional bias. The 
variability of the bias as measured by the length of a 90% confidence interval is pre- 
sented in Table 2. These lengths decrease with increasing sample size n but are not af- 


fected by the number of bootstrap replications N. 
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Figure 2. Average Bias: Average values of bootstrap estimate minus actual value 


for the true values A = 1. 
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Table 2. VARIABILITY OF BIAS: Length of the 90% confidence interval for the 
bias B. 
(Variances are less than 102 ) 


am |. 6878. |. 06% | 05888 | 681. 














ty 


(ያከ 


b. Bootstrap Variance Estimation 

The quantitv of interest here is the bias of the bootstrap estimate of the 
variance of å, Le 0“? — o?. The bootstrap estimate of the variance, o*? , is computed 
according to equation 2.7 and ce? 1s the theoretical value from equation 3.23. The average 
values of the bias of the bootstrap variance estimate are displaved graphicallv im Figure 
3 while the lengths of its 90% confidence intervals, depicting the variability, are listed in 
IE GIE. 

The graph shows that on the average bootstrap overestimates the variance 
of the maximum likelihood estimator of the scale parameter of the exponential distrib- 
ution. The average bias after some fluctuation for low values of the bootstrap repli- 
cations N seems to stabilize and from then on the number of bootstrap replications does 
not have a significant effect. Again the major impact on the bias is given by the sample 
size n. The graph cleariy shows the decrease in bias with increasing n. The variability 
of the bias of the bootstrap variance estimate, represented by the lengths of the 90% 
confidence interval of the bias also does not seem to change w:th the number of boot- 
strap replications N. Least squares regression of the lengths on the number of bootstrap 
replications vields slope parameters of the order of 105, which does not indicate a strong 


dependence. So a choice of about N = 200 bootstrap replications should be appropriate. 
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Figure 3. Bias of Bootstrap Variance Estimate: Average values of the bias of the 


bootstrap variance estimate o*? — o? for A = 1. 


Table 3. BIAS OF BOOTSTRAP VARIANCE ESTIMATE: Length of the 90% 
confidence interval of the bias of the bootstrap variance estimate 
GR — g? for) - 1. 
(Variances are less than 10^? ) 
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The results of this section, briefly summarized, are: The number of boot- 
strap replications has no major impact on the “closeness” of the bootstrap estimates to 
the theoretical values. This observation 1s in agreement with the results bv Cortes-Colon 
[Ref. $] and by Efron and Tibshirani [Ref. 7]. 


2. Confidence Intervals 
This section investigates bootstrap confidence intervals obtained bv the 


percentile and bias-corrected percentile methods. 


a. Simulation Validation 

Validation 1s an important part of every simulation. Checking the results for 
plausibility, comparing them with the theory and with results obtained by other authors 
are some of the wavs to accomplish validation. The latter way was specially chosen for 
this part of the thesis. To ensure that the percentile method and the bias-corrected 
percentile method were properly understood and correctly implemented in computer 
code, Efron's simulation [Ref. 4, page 84] was repeated. In the experiment random 
samples of size n = 15 are drawn from the exponential distribution Exp [4 = 1 ]. The 
sample are standardized to ensure that the sample mean x = 0 and the sample variance 
Y(x, —xy[(n — 1l) = 1. The bootstrap method is then applied to the standardized sam- 
ples with the number of bootstrap replications N = 1000. Selected percentiles are ap- 


proximated using the percentile method and the bias-corrected percentile method. Table 


4 shows the results for 10 trials. The averages of the estimated percentiles over the ten 
trials and the corresponding results obtained by Efron [Ref. 4, p. 85] are also presented. 
The numbers obtained are quite close to Efron's results. The simulation was pro- 
grammed in FORTRAN 77 and conducted on the NPS IBM mainframe. The random 
variates, exponential and uniform, were generated using the random number package 
LLRANDOMII [Ref. 11]. Appendix D shows the listing of the program for the 


percentile method and the bias-corrected percentile method. 


Table 4. SIMULATION VALIDATION: Nonparametric confidence intervals of 
exponential variates Exp[ « = 1], standardized, i. e. sample mean = 0 and 
sample variance = 1;n = 15, N = 1000. 


mE Percentile Method Bias-corrected PM 
5% 10% 90% 95% 5% 10% 90% 95% 


0.457 
0.425 
0.435 
0.433 
0.431 
0.451 
0.478 
0.426 
0.410 
0.404 


1 

2 

3 

4 

5 

6 

T 

8 

9 
10 


35 
| -0.36 -0.29 036 0.47 





b. Coverage 

The interpretation of a confidence interval, e. g. 90%, for a parameter of 
interest is, that in the long run with a relative frequency of 0.9, the computed confidence 
intervals cover the actual value of the parameter. Thus the relative frequency of cover- 
age can be used to assess the quality and applicability of a method, which produces 
confidence intervals. In this section, the coverage is investigated for the percentile 
method and the bias-corrected percentile method. 

The simulation looks at the central 90% confidence interval. This interval 
is set up using the Sth and 95th quantiles of the empirical bootstrap distribution for the 
scale parameter / of the exponential distribution for both methods. The simulation was 
programmed in FORTRAN 77 and run on the NPS mainframe computer. Random 


numbers were generated with LLRANDOMII [Ref. 11]. For each combination of 


sample size n and numoer of bootstrap replications N the simulation consists of 1000 
repetitions, for each of which the coverage of the actual value 2 = 1 was checked. Table 
5 shows the counts for the percentile method and Table 6 for the bias-corrected 


percentile method. 


Table 5. COVERAGE--PERCENTILE METHOD CONFIDENCE 
INTERVAL: Coverage of the true value i = 1 by the 90% confidence 
interval obtained from the percentile method, out of 1000 repetitions. 





Table 6. COVERAGE--BIAS-CORRECTED PERCENTILE METHOD CONFI- 
DENCE INTERVAL: Coverage of the true value 4 — 1 bv the 90^, 
confidence interval obtained from the bias-corrected percentile method, 
out of 1000 repetitions. 





The coverage in all cases is below the nominal level of 90%. The coverage 


appears to be somewhat erratic for the smaller values of sample sizes, n = 10 and 20, 
but it seems to improve with increasing n. Schenker [Ref. 6] observed a similar behavior 
in his investigation dealing with the estimation of the variance of a normal distribution. 
The number of bootstrap replications N again seems not to have a significant effect. 
Significant differences between the percentile method and the bias-corrected percentile 


method are also not detectable in this experiment. 


c. Percentiles 

The simulation in the previous section was set up to also provide the aver- 
age values of the Sth, 10th, 90th and 95th percentile of the empirical bootstrap distrib- 
ution. Table 7 lists these values for N = 500 bootstrap replicauons; these are averages 
of 1000 trials. Both methods, percentile and bias-corrected percentile method on the 
average overestimate the percentiles compared to the theoretical values from Table 1. 
The amount of overestimation is shown in the table in parentheses. This amount is in 
general larger for the percentile method than for the bias-corrected percentile method, 
which means that the correction, which the latter method applies, is working in the right 
direction. The difference between theoretical values and the bootstrap-based estimates 


decreases with increasing original sample size n. 


Table 7. AVERAGE PERCENTILES: Average values for percentiles obtained 
with the percentile and the bias-corrected percentile method in 1000 trials; 
ፈ = 1, number of bootstrap replications N = 500; numbers in parentheses 
are the amount of overestimation. compared to the theoretical values. 


Percentile Method Bias-corrected PM 
5% 10% 90% 95% S 10% 90% 95% 


> 0.817 ) .. › A ie 1 
(0.115) (0.113) (0.129) (0.136) | (0.100) (0.091) (0.045) (0.047) 


0.525 1.426 1.560 00757 — 0 S050 1:359. SE 
(0.053) (0.049) (0.051) | (0.040) (0.036) (0.012) (0.008) 


0.846 1.327 1424 DU SS OESO) ٣ 
(0.039) (0.035) (0.035) | (0.026) (0.024) (0.008) (0.006) 


0.557 9 +٤ ٤8 0.802 0.844 1.250 1.328 
(0.013) (0.025) (0.025) | (-0.002) (0.000) (0.005) (0.003) 


0.872 1.240 1.308 OS 0.565990] 0225 |! 
(0.028) (0.026) (0.025) | (0.020) (0.019) (0.011) (0.009) 





The behavior of percentile estimates was investigated further. The simu- 
lation for this purpose was done in SIMTBED [Ref. 9] on the author's personal com- 
puter. The 5th and 95th percentiles were selected as representative objects for 
investigation. The number of trials is 1200 for n = 10, 600 for n = 20, 480 for n = 30, 
300 for n = 40 and 240 for n = 50. Appendix A lists the results for the percentile 
method. These results show that both the standard deviation of the percentile estimate 


and the width of the central 90% confidence interval decrease with increasing sample 
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size n. The number of bootstrap replications N seems not to affect the results. Least 
squares regression of the values on the number of bootstrap replications resulted in 
values for the slope of 10-* and less. And tests for distributional fit in GRAFSTAT 
[Ref. 10] did not show significant differences for different numbers of bootstrap repli- 
cations. 

The simulation was repeated for the 5th and 95th percentiles using the 
bias-corrected percentile method. The results are listed in Appendix B. The conclusions 
for this method are basically the same as with the percentile method, decreasing standard 
deviation and width of the central 90% confidence interval with increasing sample size 
and no effect of the number of bootstrap replications. The only difference again is that 
the bias-corrected percentile method is on the average closer to the theoretical value 


than the percentile method. 
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IV. NORMAL LINEAR REGRESSION 


The results of simulations to study the properties of bootstrap estimators of the 


parameters in a simple linear regression model are presented in this chapter. 


A. THEORETICAL OVERVIEW 
l. The Regression Model 
Let f(x,,94 (5. S, Ys y) be n pairs ol observations with as sela 
dependent variable and y the dependent variable. Under the assumptions of independ- 
ence, normal distribution and homoscedasticity for the random variables Y,, the model 


for a linear relationship between x and Y, is 

Y, = Bo + Bese, Jfori Ia. (4.25) 
The random variables e, have mean 0 and variance o? and are normally distributed: 

c; ^ N(0,o) fori 5 1, 2,3,..., n. (4.26) 


It is well known that the maximum likelihood estimates for the coefficients fj, and fj, are 


n 


XV; = HX V 


)4.27( = سے ہم 
nx‏ — 2 
i=]‏ 

and 

Bg — y — fx. (4.28) 


Both estimators are unbiased, i. e. 


ELB] = Bo and Ef] = fi (4.29) 


The joint sampling distribution for fj, and £, is known to be a bivariate normal distrib- 
ution. The marginal distributions are normal with means equal to the respective true 


values and the variances 
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EY 





Var[fg] = Sr — (4.30) 


and 


Varth] - SEE. (4.31) 


? (x = a 


| 


The covariance between fj, and f, is 


27 
Y 


O . 
n 
፡ህ = xy 
መፅ 


Covlho, fı] = - (4.32) 


2. Bootstrap Method for Regression Models 
The implementation of the bootstrap method for regression models [Ref. 4] 
differs slightly from the one in the one-sample case. It 1s described here for the normal 
linear regression of one dependent and one independent variable, which is the topic of 
this chapter. 
To perform the bootstrap, first the least squares estimates B, and 7 (equations 


4.27 and 4.28) are computed. These estimates are used to compute the residuals e, : 


EE fa) or i — 1,2, 3, 7. (4.33) 


A bootstrap sample e" of size n, which is of the same size as the original sample, is 


obtained by randomly drawing with replacement n times from the e. Computing 
7 A A 
y= Bo + bin ket fori — 1,2, n (4.34) 


ie ON ar bias of observations 1131,7' 1]1፡25,7 3/4297 00. Y*.)) . These n pairs 
of ‘observations’ are used to compute the bootstrap estimates ß*, and ß*, using the 
equations 4.27 and 4.28. The process of randoniy drawing and computing the estimates 


is repeated for a total of N bootstrap replications. The bootstrap estimates 6”, and 
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B*,, for, = 1, 2, 3, .... N are used.to eonstmiet the 81 +۰ 
the estimators f, and f,. All other quantities then can be estimated as described in 


Chapter iit 


B. THE SIMULATIONS 

The choice of the values for x, the independent variable in the regression model in- 
fluences the variability of the results. Since this effect was not to be investigated, the 
values for x were kept fixed throughout the simulations. The values for the x were evenly 
spread from 10;n to 10 in increments of 10/n where n indicates the sample size. The 
values chosen for the coefficients fj, and fj, and the variance c? are discussed below. For 


each simulation the sample pairs (x,y) where obtained by computing 
y = Bo + Bx, FE, foriz1,2,3,..,n, (4.35) 


with the e, being normal random variates with mean zero and variance ወ". Having set 
up the pairs of observations, the simulation then proceeds as described in the previous 


section. 


l. Estimation of the Regression Coefficients 

The first simulation estimated the coefficients fj, and fj, by applying equations 
4.27 and 4.28 to each bootstrap sample and averaging over N, the number of bootstrap 
replications. It was conducted in SIMTBED [Ref. 9] on the author's personal computer 
and consisted of ten super replications. For n = 10, 20, 30, 40 and 50 the respective 
number of trials was 300, 150, 100, 75 and 60 within each super replication. This sums 
up to a total number of trials of 3000 for n = 10, 1500 for n = 20, 1000 for n ^ 30 etc.. 
ላ simulation with those ten super replications was conducted for each selected number 
of bootstrap replications N. For this simulation the values of the parameters were fi, 
= 1.5, ß, = 0.8 and o? = 0.5. The tables below show the averages of the estimates over 


all super replications, Table 8 for pa, and Table 9 for ፆ*.. 
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Table 8. BOOTSTRAP ESTIMATE: Y-INTERCEPT: Average values and 
standard deviations ( ) of the bootstrap estimate of the v-intercept; actual 
ES — ue es — 









1. ا‎ 1 _ 1.487 1. E 1.492 
(0.4782) (0.3348) (0.2657) (0.2286) (0.2073) 
1.507 1.501 1.491 1.500 1.514 
(0.4835) (0.3253) (0.2692) (0.2264) (0.2043) 
- 1.497 1.485 1.492 1.498 1.506 
= (0.4777) (0.3224) (0.2588) (0.2283) (0.2017) 
e 1.489 1.506 1.504 1.493 1.496 
(0.4848) (0.3390) (0.2586) (0.2316) (0.1961) 
A. 1.492 1.519 1.482 1.493 1.509 
(0.4799) (0.3226) (0.2715) (0.2288) (0.2054) 
m 1,507 1.199 1.509 1.508 1.504 
: (0.4838) (0.3251) (0.2605) (0.2292) Eo 





Table 9. BOOTSTRAP ESTIMATE: SLOPE: Average values and standard de- 
[0 x ) of the bootstrap estimate of the slope parameter; actual value 


ET 





ma EG ro 20 | 
0.7991 0.8006 0.8011 0.7990 0.8006 
(0.07755) | (0.05575) | (0.04460) | (0.03988) | (0.03353) 
0.7981 0.7997 0.8011 0.8002 0.7985 
(0.07837) ን 05456) | (0.04596) | (0.03935) | (0.03502) 
0.8003 0.8029 0.8015 0.8012 0.7985 
(0.07665) | (0.05315) | (0.04503) | (0.03845) | (0.03538) 
0.8002 0.7986 0.8006 0.8011 0.7992 
(0.07778) | (0.05561) | (0.04452) | (0.03958) | (0.03383) 
0.8008 0.7969 0.8027 0.8017 0.7986 
(0.07835) | (0.08486) | (0.04690) | (0.03950) | (0.03563) 
1 0.7989 0.8008 0.7987 0.7987 0.7995 
(0.07765) | (0.05451) | (0.04395) | (0.03948) | (0.03126) 


It can be seen that on the average the bootstrap estimates of the regression pa- 






= 














rameters are fairly close to the theoretical values. The number of bootstrap replications 


N again seems not to affect the "closeness" of the bootstrap estimates to the theoretical 
values. The computed standard deviations of the estimates (shown in parentheses) also 
confirm this conclusion. As a more detailed representation of the variability, 
Appendix C shows selected quantiles of the bootstrap estimates of the regression coef- 
ficients from the simulation with N = 300. The simulation results compare favorably 
with the theory. 

For selected numbers of bootstrap replications the simulation was repeated with 
different sets of values for the parameters f,, f, and o?. The following tables show the 


simulation results. 


Table 10. ESTIMATION OF THE REGRESSION COEFFICIENTS: Average 
values and standard deviations of the bootstrap estimates of the re- 
gression coefhicients; N = 300; theoretical values: 6, = 0.5, B, = 


O oo 
ሚሙ መሚ OE OE 


0.4893 0.5055 0.5041 0.4929 0.4957 
(0.3848) (0.5390) (0.2586) (052516) (0.1961) 






2.000 0 2.001 2.001 (99) 
(0.0778) (0.0556) (0.0445) (0.0396) (0.0338) 


Table 11. ESTIMATION OF THE REGRESSION COEFFICIENTS: Average 
values and standard deviations of the bootstrap estimates of the re- 
gression coefficients, N = 200; theoretical values: f, = 1.5, f, = 0.8, 
u = a a 





1. سے ان سے 1.457 ا‎ 477 l. —— 1. a 

(1.3310) )0.9119( (0 7319) (0.6458) (0.5705) 

Á 0.8009 0.8082 0.8042 0.8033 0.7958 
$ 1 (0.2168) (0.1505) (0.1274) (0.1087) (0.1001) 


The averages of the bootstrap estimates are again quite close to the actual values. The 
changes ın the standard deviations clearly reflect the changes in the parameter values 


and conform with the theory. 
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2. Bootstrap Estimates of the Variances 

The same setup, as far as the number of repetitions and actual values for the 
parameters are concerned, was used for this simulation. The quantities under investi- 
gation now were the differences between the bootstrap estimates and the theoretical 
values of the variances and the covariance of the least squares estimators. The bootstrap 
estimates for the variances and the covariance were obtained following equation 2.7, 
while the theoretical values were computed according to equations 4.30, 4.31 and 4.32. 
The average differences for the variances turned out to be negative, i. e. the bootstrap 
estimate is on the average lower than the theoretical value. For the covariance which is 
negative the average differences were positive Which means that the absolute value of the 
bootstrap covariance estimate on the average is lower than the theoretical value. The 
average absolute values of the differences were less than 0.05 and decreasing with in- 
creasing original sample size n. The number of bootstrap replications N had no effect 
on the average difference. Standard deviations for the differences were of the order 0.1, 
decreasing with increasing n. 

Again for some selected cases the simulation was repeated with different sets of 
values for the parameters f,. ß, and a”. The following Table 12 shows the results of one 


of these. 


Table 12. BOOTSTRAP VARIANCE ESTIMATE: Average values and standard 


deviations of the difference AVar*[f] between the bootstrap variance es- 
umates of the regression coeflicients and the theoretical value; N = 200; 


theoretical values: f, — 1.5. fi, = 0.5, 7" = 4.0. 


anl mm Tm | Le 
am -0,3605 -0.0921 -0.0310 -0.0271 -0.0124 
اس تد‎ | (0,7596) (0.2765) (0.1509) (0.0970) (0.0732) 


The bootstrap estimates of the variance of fj, and the covariance between fj, and 








pf, were also quite accurate; the differences from the corresponding theoretical values 


were of the order 0.05 with standard deviations of the order 0.1. 


3. Confidence Intervals 
In this section the bootstrap estimates for percentiles as bounds for confidence 


intervals for the estimates of the regression coefficients are investigated. The investi- 


25 


gation focuses on the coverage of the central 90% confidence interval using both the 
percentile and the bias-corrected percentile method. The simulation was written in 
FORTRAN 77 for the IBM mainframe computer. It used the theoretical values 
ß, L5,f, = 0.8 and o? = 0.5. 500 trials were used for each selected combination 
of sample size n and bootstrap replications N. Tables 13 and 14 lists the results for the 
percentile method and Tables 15 and 16 for the bias-corrected percentile method. The 
coverage for both methods is in most cases below the nominal 90% level although the 
differences are fairly small. The increase in coverage with increasing sample size n seems 
obvious while the number of bootstrap replications N does not seem to have any influ- 
ence. A significant difference between the two methods can not be demonstrated with 


the results. 


Table 13. COVERAGE  - PERCENTILE METHOD CONFIDENCE 
INTERVAL: Percentage of coverage of the true value f, = 1.5 bv the 
bootstrap 90% confidence interval obtained from the percentile method, 
in 500 trials. 





Table 14. COVERAGE  - PERCENTILE METHOD CONFIDENCE 
INTERVAL: Percentage of coverage of the true value fj, = 0.8 by the 
bootstrap 90% confidence interval obtained from the percentile method, 
in 500 trials. 
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Table 15. COVERAGE--BIAS-CORRECTED PERCENTILE METHOD CONFI- 
DENCE INTERVAL: Percentage of coverage of the true value fj, ^ 1.5 
by the bootstrap 90% confidence interval obtained from the bias- 
corrected percentile method, out of 500 repetitions. 





Table 16. COVERAGE--BIAS-CORRECTED PERCENTILE METHOD CONFI- 
DENCE INTERVAL: Percentage ol coverage of the true value f, = 0.8 
by the bootstrap 90% confidence interval obtained from the bias- 
corrected percentile method, in 500 trials. 





4. Linear Regression with Mixtures of Normals 


As a further test case for the bootstrap of regression models, another linear re- 
gression model was chosen. In this model all the assumptions about the dependent ran- 
dom variable Y as in the previous model hold, except the distributional assumption. 
Now the underlying distribution is assumed to be a mixture of two normal distributions. 


The model equation 4.25 still holds, but now equation 4.26 becomes 


\(u,,0.) with probability 
نہ رع‎ | Uum) 4 vale fori = 1, 2,3 … n. (4.36) 


N(u,05) with probability (1-p) 


The expectation for e is 


Ete O (4.37) 


In the simulations E[e] is for convenience set equal to zero by adjusting the values of 
p, 4, and ዞን appropriately. The resulting variance of the mixture of two normal distrib- 


utions is 
E 2 2 2 A38 
Gres = po, + (1 — pe Sol EA S (4.38) 


The same simulation setup as for the normal linear regression in SIM TBED was 
used to conduct simulations for this regression model. The following tables, Table 17 and 


Table 18 show the results of two simulation runs. 


Table 17. ESTIMATION OF THE REGRESSION COEFFICIENTS: Average 
values and standard deviations of the bootstrap estimates of the re- 
gression coefficients; N = 100; theoretical values: f, = 1.5, f, = 0.8, 
— AJ = sS = pue ur 






: 






pu 


1. Le 1. di 1.498 n 2 pt 
Gi) (0.7922) ሐ 6319) (0.5180) on 

i 0.8052 0.8014 0.8003 0.8034 0.8048 
pr, (0.1824) (0.1317) (0.1081) (0.0885) (0.0834) 


Table 18. ESTIMATION OF THE REGRESSION COEFFICIENTS: Average 
values and standard deviations of the bootstrap estimates of the re- 
gression coefficients: N = 200; theoretical values: fà — 1.5, fi, = 0.8, 
DE jac ና = 0.23, = - 2? = 0.5, ہت‎ = 0.3, — = 1.0. 


2282 
1. m — EE اس‎ 1.500 
(0.8713) (0.5979) (0.4638) (0.4145) (0.3660) 
0.7990 0.7968 0.7980 0.8053 0.7971 
"1 (0.1397) (0.0973) (0.0798) (0.0712) (0.0620) 
The estimates for the parameters are once more close to the actual values. 


For $, = 1.5, B, = 0.8, p ^ 025, u, 9 -1.5, 6? ^ 0.5, u, 7 0.5 and ei = 1.0 


90% bootstrap confidence intervals were computed. The simulation consisted of 500 









trials for each combination of the sample size n and the bootstrap replications N. The 


following four tables, Table 19 through Table 22 contain the results of the simulation, 


Tables 19 and 20 for the percentile method and Tables 21 and 22 for the bias corrected 


percentile method. 


Table 19. 


COVERAGE  - PERCENTILE METHOD CONFIDENCE 
INKERVAL ZZ Percentage of coverage of the true value f, — 1.5 bv the 
bootstrap 90% confidence interval obtained from the percentile method, 
in 500 trials. 





lable 20. 


COVERAGE  - PERCENTILE METHOD CONFIDENCE 
INTERVAL: Percentage of coverage of the true value f, — 0.8 by the 
bootstrap 90% confidence interval obtained from the percentile method, 
in 500 trials. 





The results are almost the same as for the standard regression model. The 


coverages are with two exceptions below the nominal level; they increase with increasing 


sample size n and the bootstrap replications N again seem not to affect the coverage. 


Also no significant difference is detectable between the coverages of the confidence in- 


tervals from the percentile method and the bias-corrected percentile method. 
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Table 21. COVERAGE--BIAS-CORRECTED PERCENTILE METHOD CONFI- 
DENCE INTERVAL: Percentage of coverage of the true value fj, — 1.5 
by the bootstrap 90% confidence interv al obtained from the bias- 
corrected percentile method, out of 500 trials. 





Table 22. COVERAGE--BIAS-CORRECTED PERCENTILE METHOD CONFI- 
DENCE INTERVAL: Percentage of coverage of the true value fj, = 
by the bootstrap 90% confidence interval obtained from the bias- 
corrected percentile method, in 500 trials. 
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V. CONCLUSIONS 


The subject of this thesis is an investigation of the performance of the bootstrap 
method for small sample sizes in the two scenarios of the exponential distribution and 
linear regression. In both cases the simulation results show that the bootstrap method 
provides reasonable approximations for the estimation of statistical parameters. 

The simulations clearly show that the sample size n has the most impact on the ac- 
curacy of bootstrap estimates. Once the sample size is fixed, the “goodness” of the 
bootstrap estimator 1s essentially constant independent of the number of bootstrap rep- 
hcations N, provided that N is above a minimum value which in this investigation turned 
out to be about 200. 

For the estimation of a parameter Ó , given an estimator 0 = A(X) , the bootstrap 
estimate 6* does not seem to have an edge over the conventional estimate 0. In the 
linear regression simulations the bootstrap point estimates of the regression coeflicients 
are on the average very close to their actual values and for the normal linear regression 
their distributions approximate the theoretical normal distributions. For the exponential 
distribution the simulated bootstrap estimates for the scale parameter showed an average 
bias which 1s significantly larger than the bias predicted bv the theory for the maximum 
likelihood estimator. So for the point estimation of the parameter the maximum likeli- 
hood estimator appears to be the better estimator and the extra effort of going through 
the process of the bootstrap method does not bring any improvement. 

For the estimation of the variance of an estimator, bootstrap in both investigated 
scenarios provides estimates which are very close to the theoretical results. While in the 
case of the exponential distribution the bootstrap estimator on the average slightly 
overestimates the variance, for the normal linear regression problem the bootstrap esti- 
mator 1s slightly below the theoretical value. 

The coverage of bootstrap confidence intervals is below the nominal level for both 
the percentile method and the bias-corrected percentile method and not significantly 
different between the exponential and the linear regression cases. The latter do not show 
significant differences for the different distributions, viz., normal distribution or mixture 
of normal distributions. For small sample sizes (n = 10 and 20) the low coverage seems 
to indicate that the resulting confidence intervals are of little use. However for n = 40 


or 50, the coverage increases and approaches the nominal level. 
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An important question in practical applications is, how manv bootstrap replications 
should be taken. The simulations show that for the estimation of the variance of the 
maximum likelihood estimator of the scale parameter of the exponential distribution 200 
bootstrap replications are sufficient for all sample sizes. For the higher sample sizes (n 
= 40, 50) even less bootstrap replications (N = 50, 100) produce results of similar ac- 
curacy. For the estimation of the variance of the estimator for the coefficients in the 
linear regression, the simulations show that 100 bootstrap replications provide reason- 
able estimates. The simulations for both scenarios show that increasing the number of 
bootstrap replications bevond the values indicated does not significantly increase the 
quality of the results. For the estimation of confidence intervals the answer 1s not as 
straightforward as for the variance. If the coverage of the actual value of the parameter 
by the confidence interval 15 taken as a measure for the quality of the confidence interval 
estimate, the simulation results for both scenarios do not show any significant influence 
of the number of bootstrap replications. The investigations of the percentiles for the 
exponential distribution also confirm this conclusion. While Efron and Tibshiram 
[Ref. 7] and Efron [Ref. 12] state that for confidence intervals a minimum of 1000 
bootstrap replications 1s required, the simulations in the special case of the exponential 
distribution indicate that 400 bootstrap replications would be sufficient to obtain rea- 


sonable confidence interval estimates. 
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APPENDIX A. PERCENTILE ESTIMATION--PERCENTILE METHOD 


Table 23. STH PERCENTILE: Average values, standard deviations ( ) and 
lengths of the 90% ‘0 p interval (( )) for bootstrap estimates of the 


D of 4* E‏ کے کچ 


Ta Ta Ts‏ یت 














ee 7531 0.7789 0.7955 0.8067 0.8233 
(0.2712) (0.1959) (0.1516) (0.1332) (0.1193) 
do 8209)) | ((0.6157)) | ((0.5052)) | ((0.4528)) | ((0.4211)) 






















0.7581 0.7648 09" 0.8093 0.8347 
(0.2709) (0.1880) (0.1418) (0.1317) (021255) 
((0.8604)) ((0.6375)) ((0.4861)) ((0.40-44)) ((0.4410)) 
0.7582 oT 0.7970 0.8299 0.8167 


(0.2758) (0.1792) (0.1511) (0.1414) (0.1172) 
((0.8291)) | ((0.5933)) | ((0.5074)) | ((0.4964)) | ((0.3813)) 


LS. 
© 
© 





























0.7620 0.7597 0,7935 0.8085 0.8361 
(0.2650) (0.1817) (0.1388) (0.1351) (0.1294) 
lS) ((0.6060)) ((0.4443)) ((0.4574)) ((0.4323)) 
0.7514 0.7793 0.8013 0.8212 0.8170 
1000 (0.2708) (0.1845) (0.1569) (0.1385) (0.1345) 
((0.8424)) ((0.5913)) ((0.5032)) ((0.4659)) ((0.4539)) 
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Table 24. 95TH PERCENTILE: Average values, standard deviations ( ) and 


lengths of the 90% o confidence interval (( )) for bootstrap estimates of the 
— A of ás Adel. 


l. a 
(0.8067) 
((2.424)) 

1.996 
(0.8235) 
((2.501)) 

1.999 
(0.8209) 
((2.625)) 


ና 
(0,7518) 
((2.356)) 


1.996 
(o 7879) 
,568)) 


20 
1.574 


(0.4163) 
[[1.272)) 


1.549 
(0.3944) 


((1.341)) 
1.536 
(0.3721) 
((1.264)) 
1.534 
(0.3775) 
((1.299)) 


1.571 
(0.3740) 
((1.259)) 
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1.419 
(0.2754) 
((0.928)) 


1.415 
(0.2626) 
((0.873)) 


1.428 
(0.2831) 
((0.928)) 


1.425 
(0.2681) 
((0.963)) 


1.437 
(0.2875) 
((0.963)) 


1:325 
(0.2173) 
((0.745)) 
1.337 
(0222) 
((0.748)) 


1.369 
(0.2389) 
((0.807)) 


1.343 
(0.2284) 
((0.747)) 
1.368 
(0.2358) 
((0.780)) 
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1.290 
(0.1895) 
((0.656)) 


1.315 
(0.1886) 
((0.670)) 


1.292 
(0.1802) 
((0.644)) 


1.311 
(0.1956) 
((0.678)) 
1.288 
(0.2141) 
((0.764)) 





APPENDIX B. PERCENTILE ESTIMATION--BIAS-CORRECTED 
PERCENTILE METHOD 


Table 25. STH PERCENTILE: Average values, standard deviations ( ) and 
lengths of the 90?» confidence interval (( )) for bootstrap estimates of the 


MT P of it i = 1. 


0. — — 000٦ 008۷ 0.7946 0.8155 


(0.2734) (0.2004) (0.1564) (0.1318) (0.1194) 
((0.844)) ((0.635)) ((0.530)) ((0.445)) ((0.432)) 


0.7404 0.7444 0.7819 0.7979 Dep? 
(0.2736) (0.1823) (0.1520) (OI, (0.1214) 
((0.834)) ((0.615)) ((0.494)) ((0.477)) ((0.418)) 


0.7381 0.7396 0.7858 O 02S 0.8073 
(0.2708) (0.1692) (0.1491) (0.1363) (0.1154) 
((0.828)) ((0.557)) ((0.510)) ((0.472)) ((0.393)) 

0.7524 0.7452 887733 0.8040 0.8297 
(0.26--4) (0.1751) (0.1362) (0.1300) )0.1281( 
((0.817)) ((0.586)) ((0.422)) ((0.419)) ((0.435)) 

0.7251 0.7680 0.7879 0.8117 0.8050 
(0.2586) (0.1851) (0.1581) (0.1395) (0.1299) 
((0.802)) ((0.591)) ((0.500)) ((0.461)) ((0.430)) 
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Table 26. 95TH PERCENTILE: Average values, standard deviations ( ) and 
lengths of the 90? fo confidence interval (( (O) for bootstrap estimates of the 


ur AR of ?* 200م‎ Ll, 


l. — 
(0.7564) 
wen 


1.900 
(0.7534) 
(2.321)) 


1.908 
(0.7625) 
((2.176)) 


1.911 
(0.7304) 
(2.350) 
1.873 
(0.6977) 


MEI, 


1.522 
(0.3970) 
((1.201)) 


1.504 
(0.3732) 
((1.267)) 


1.492 
(0.3537) 
((1.193)) 
1.419 
(0.3595) 
((1.235)) 
1.529 
(0.3568) 


) 1) 
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1.386 
(0.2683) 
((0.934)) 


1,387 
(0.2569) 
((0.868)) 


1.401 
(0.2733) 
((0.897)) 


1.396 
(0.2574) 
((0.892)) 


1.408 
(0.2781) 


((0.915)) 


1.305 
(02151) 
((0.749)) 


1.361 
(0.2171) 
((0.774)) 


1.350 
(0.2345) 
((0.818)) 


1.322 
(0.2265) 
((0.739)) 


1.345 


(0.2290) 
((0.764)) 


1.273 
(0.1888) 
((0.653)) 


1.297 
(0.1849) 
((0.658)) 


1.276 
(0.1785) 
((0.638)) 


1.259 
(0.1942) 
((0.665) 


1.272 
(0.2084) 
((0.108)) | 





APPENDIX C. VARIABILITY OF BOOTSTRAP POINT 
ESTIMATES--LINEAR REGRESSION 


Table 27. VARIABILITY OF THE BOOTSTRAP ESTIMATE: 
Y-INTERCEPT:  Quantiles for the bootstrap estimate of the y- 
intercept, compared to the normal quantiles (in parentheses); theoretical 
IE ከሮ ስቄ ን 


Quante n) 0 | 2 | 3 | o | 
0.3409 0.7080 0.8260 0.9660 1.023 
(0.3763) (0.7359) (0.8840) (0.9699) (1.0277) 
0.5456 0.8576 0.9591 1.018 1.062 
(0.5532) (0.8562) (0.9810) (1.0534) (11021) 
0.050 0.7194 0.9476 1.043 IE ¡SIS 
dl (0.7055) (0.9597) (1.0645) (1.1252) (1.1660) 
0.100 0.8812 1.063 1.154 1.218 27 
(0.8810) (1.0790) (1.1607) (1.2080) (1.2398) 
50 1.173 1.266 Lae 1.338 1.368 
i (1.1742) (1.2784) (1.3214) (1.3463) (1.3631) 
1.500 1.475 1.485 1.498 | በ! 
— 5000) (1.5000) (1.5000) (1.5000) (1.5000) 
750 1.700 1.665 1.649 1.644 
a Y ss (17216) (1.6786) (1.6537) (1.6269) 
0.900 21002 1.903 s: 830 1.794 1.769 
(2.1190) a. 9210) .8393) ቦ. 7920) (1.7602) 
0.950 2.215 2.042 " 934 1.907 1.845 
ነው (2.2945) (2.0403) (1.9355) (1.8748) (1.8349) 
0.975 2.449 2.160 2.035 1.982 1.916 
(2.4468) (2.1438) (2.0190) (1.9466) (1.8979) 
0.990 2.607 2.289 221119 ከ. 1.946 
| o (2.2641) (2.1160) (2.0300) (1.9723) 












B 
c 








= 
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Table 28. VARIABILITY OF THE BOOTSTRAP ESTIMATE: SLOPE: 
Quanules for the bootstrap estimate of the slope, compared to the normal! 
quantiles (in o theoretical value a ME = 0.8, 


069% -8[. m | m 
0.6161 0.6727 —2— “ረሙ = 
(0.6189) (0.6724) 0 6959) (0.7099) (0.7193) 
nee 0.6471 0.6963 0.7085 0.7185 0.7195 
ics (0.6474) (0.6925) (0.7123) (0.7241) (0.7321) 
m 0.6729 0.7179 0.7237 0.7358 0.7352 
HE (0.6719) (0.7098) (0.7264) (0.7363) (0.7430) 
0.7022 0.7347 0.7427 0.7520 0.7510 
(0.7002) (0.7297) (0.7427) (0.7504) (0.7556) 
m 0.7472 0.7662 0.7702 0.7770 0.7752 
(0.7475) (0.7630) (0.7698) (0.7739) (0.7766) 


0.500 0.8008 0.8027 0.8021 OE 0.8005 
7 (0.8000) (0.8000) (0.8000) (0.8000) (0.8000) 
0.750 0.8503 0.8388 0.8333 0.8266 0.8220 
A (0.8525) (0.8370) (0.8302) (0.8261) (0.8234) 
0.900 0.8988 0.8713 0.8584 0.8513 0.8436 
(0.8998) (0.8703) (0.8573) (0.8496) (0.8444) 
0.9298 0.8919 0.8771 0.8663 0.8600 

(0.9281) (0.8902) (0.8736) (0.8637) (0.8570) 

0.973 0.9580 QoS 0.8954 0.881. 0.8736 
(0.9526) (0.9075) (0.8877) (0.8759) (0.8679) 


0,990 0.9759 0.9362 0.9138 0: 8927 0.5807 
(0.9811) (0.9276) (0.904) ) (0.8901) (0.5806) 
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APPENDIX D. FORTRAN PROGRAM FOR BOOTSTRAP 


The program listed here, TEST, was written in the development of the simulations 
for this thesis. Its primary purpose is the validation of the simulations by repeating 
Efron's experiment [Ref. 4, pp. 84] as described in Chapter III. It is listed here as an 
example for how the bootstrap method, the percentile method and the bias-corrected 
percentile method were implemented in the various simulations. The program is written 
in FORTRAN 77 and designed to run under the IBM VM/CMS operating system. 

The main program contains the generation of the original sample, the generation 
of the bootstrap samples and the computation of the estimator for each bootstrap sam- 
ple. After sorting the bootstrap estimates the percentiles are computed with the 
percentile method and the bias-corrected percentile method. 

The subroutine SHELLS 1s an implementation of the SHELL sort algorithm. 

The subroutine NORMAL computes probabilities for the standard normal distrib- 


ution based on the following approximation formula [Ref. 13, p. 932] 
‹0(2) —- a 


=1 -- 0,5 ።(1 + Az + BERG Dz” Ab bz? ቴፔ Fz°) Wor 2 20 


with A = 0.0498673470 
B = 0.0211410061 
C = 0.0032776263 
D = 0.0000380036 
E = 0.0000488906 


F = 0.0000053830. 
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The subroutine INVNOR computes the quantiles for the standard normal distrib- 


ution using the approximation formula [Ref. 13, p. 933] 


Zp = q^ (p) 


ABT UE 
Ia DI ET ae 


with 


P= In: for p 2 0.5 


2.515358] 
0.802853 
0.010328 
1.432788 
E = 0.189269 


F = 0.001308 
The subroutines NORMAL and INVNOR are both used for the bias-corrected 


and 


Sao) SU 
٢ ٢ 


percentile method. 
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PROGRAM TEST 


Ee EE ev eer so o o o le ee cle ate aie nin ote sace ከ 9 - 9. 9. p. 0. E 29 4. e کڈ ات‎ eich e o de 
ኔ 


Program to verify implementation of methods by repeating Efron's 
simulation for comparison. + 


Simulate the bootstrap from an Exponential distribution with . * 
parameter LAMBDA with standardized original samples. ዬ 
Compute 5th, 10th, 90th and 95th percentiles using the * 


ve 

* - Percentile Method * 
* - Bias-Corrected Percentile Method * 
ele ete 
* Variables and parameters: * 
* N Original sample size = 15 E 
E NN Number of bootstrap replications = 1000 * 
* M Number of repetitions = 10 * 
© ORIG the original sample * 
* RAND the Uniform (0,1) random numbers for the bootstrap * 
* DRAW the integer random numbers for the bootstrap * 
* LHAT the vector of MLEs of the bootstraps * 
FSFE TE TOTS jede ye ve eve deve e dede ede e deve ee eee vede eee لو و مو مود چاو پل کو پا کو کو کو ہو کاو ہو کو کے چو پاو ںاج چو ڈو ڈو کو و ہو چو او‎ ESE کا‎ 


* Declare variables and I/O devices 


INTEGER N, NN, M, IX1, IX2, ISORT, MUL1, MUL2, DRAW, LOOK 
PARAMETER (N=15, NN=1000, M=10) 

REAL LAMBDA, AV5, AV10, AV90, AV95, CDFLHA, ZPRIME, Z5, Z10, Z90, 
CZ95, AUX5, AUX10, AUX90, AUX95, BAVS, BAV10, BAV90, BAV95, 
CAAAS, AAA10, AAA90, AAA95 

PARAMETER (LAMBDA=1. 0) 

REAL ORIG(N), RAND(N), LHAT(NN), P5(M), P10(M), P90(M), P95(M), 
CBCPS(M), BCP10(M), BCP90(M), BCP95(M) 

DATA IX1/31397/, 1X2/75931/, MUL1/1/, MUL2/2/, ISORT/O/, AV5/0/, 
CAV10/0/, AV90/0/, AV95/0/, BAV5/0/, BAV10/0/, BAV90/0/, BAV95/0/ 


CALL EXCMS('FILEDEF 10 DISK OUTEST LISTING A') 


* Output header and compute constants 


90 


WRLIECIO;90) N, NN 

Omer 1 / © TOR, BOOTSTRAP SIMULATION / 0’ ,10X, 
C'Nonparametric confidence intervals for the expectation, '/11X, 
C'negative exponential distribution; /11X, 


“N sEandardized samples of size m= ,14/11X, 

eenumber ot bootstrap replications “N = ',I6/ 0 ,4X, 

Oral Percentile Meth. Bias-corr. Percentile ' 
C Meth. /5X, 

C 5% 10% 90% 95% 5% 10% 90% | 


ን ን ገ ወን‏ “مم 


CALL INVNOR(O. 95,25) 
CALL INVNOR(0. 9,210) 
CALL INVNOR(O. 1,290) 
CALL INVNOR(O. 05,295) 


* Bor M repetitions 
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DO. 30 Ke pu 
** Create the original sample 
CALL LEXPN(IX1,O0RIG,N,MUL1,ISORT) 


oe 


Standardize the original sample 


A= 0 
B = 0 
DO T0, 2 JU. z& l1 8. 
A = A + ORIG(JJ) 
B = B + ORIG(JJ)**2 
10 CONTINUE 


SD = SQRT((B - A*A/N)/(N-1)) 
DO 11, JJ = 1, N 
ORIG(JJ) = (ORIG(JJ) - A/N)/SD 
ab] CONTINUE 


** Do NN bootstrap replications 
DO Po Ted NN 
CALL LRND(IX2,RAND,N,MUL2,ISORT) 


LHAT(I) = 0 


DO AU 4 Lo, N 


DRAW = INT(N*RAND(J)) + 1 
LHAT(I) = LHAT(I) + ORIG(DRAW)/N 
27 CONTINUE 
20 CONTINUE 


** Sort the bootstrap estimates 
CALL SHELLS(LHAT,NN) 
** Compute percentiles using the percentile method 


P5(K) = LHAT(50) 


P10(K) » LHAT( 100) 
P90(K) = LHAT(900) 
P95(K) = LHAT(950) 


xx Compute the percentiles using the bias-corrcted percentile method 


LOOK = NN/2 
111 IF(LHAT( LOOK). GT. 0. AND. LHAT( LOOK*1). GT. 0) THEN 
LOOK = LOOK - 1 
EG do) ln 


ELSE IF(LHAT(LOOK). LT. 0. AND. LHAT(LOOK+1).LT.0) THEN 
LOOK = LOOK + 1 
۶ 71 

END IF 


CDFLHA = REAL(LOOK) /NN 
CALL INVNOR(CDFLHA,ZPRIME) 
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AUS = 257۲۲181 = 45 

AUX10 2.2 6 0ۃ‎ 

AUX90 2 26 0 

BONUS —U2*4PRIME > 293 

CALL NORMAL(AUX5,AAA5) 

CALL NORMAL(AUX10,AAA10) 
CALL NORMAL(AUX90,AAA90) 
CALL NORMAL(AUX95,AAA95) 
BCP5(K) = LHAT( INT(AAA5*NN)) 


BCP10(K) = LHAT(INT(AAA1O*NN)) 
BCP90(K) 7» LHAT( INT(AAA90*NN)) 
BCP95(K) 72 LHAT( INT( AAA95*NN)) 


** Output trial results 


WRITE( 10,91) K,P5(K), P10(K), P90(K), P95(K), BCP5(K), 
C  BCP10(K), BCP90(K), BCP95(K) 
91 FORMAT('0',18,4(2X,F6.3),3X,4(2X,F6. 3)) 
30 CONTINUE 


* Compute and output averages 


DO 55 K sd, M 
AV5 = AV5 + P5(K)/M 


AV10 = AV10 + P10(K)/M 
AV90 = AV90 + P90(K) /M 
AV95 = AV95 + P95(K)/M 
BAV5 = BAV5 + BCP5(K)/M 
BAV10 = BAV10 + BCP10(K)/M 
BAV90 = BAV90 + BCP90(K)/M 
BAV95 = BAV95 + BCP95(K)/M 


55 CONTINUE 

WRITE( 10,92) AVS, AV10, AV90, AV95, BAV5, BAV10, BAV90, BAV95 
92 FORMAT('0',1X, Average’ ,4(2X,F6. 3) ,3X,4(2X,F6. 3)) 

STOP 

END 


ሃኝሮንኛንኞንሮዮንኝሮንሮንኞንጀንጅኞኝሮንኛኝሮንኛ سار حا سار‎ ESE ESE سا حا‎ ESE SE SE EI EE SE EK EE IE ste nee ESE EE SEE ETE EE le le te sle te lele se Ee ST IE 


SUBROUTINE SHELLS(UNSORT, NUM) 


ste Je gez Yale Jeu ey. elen ele Je ste 'ese ve Je Je Ja Je Je Je Je Je Jon sle Je Je Je Je Je #2 dentado ےگ ول مک‎ te dee Jes sie sesse seve Je Jes tata Je ele eeen Je eves'es'c sevevevevesove 


# Subroutine SHELLS to sort data in 8 ۳ order ር sort) * 


Jelena dan sese ےکن ےکن ےن ے‎ a nt daras dara tanda s اح ما مگ حا‎ te من ان ی لن ےل ےا‎ ows See's adenda ton چاو چو لو ےلو او او‎ de le le lade ese se ses ooste ees ese ese se 


INTEGER NUM, GAP, COUNT 
REAL UNSORT(NUM) 


GAP = NUM 
10 GAP = INT(CAP/2. 0) 
20 | COUNT 2 0 
DO 40, I = 1, NUM - GAP 
IF (UNSORT(1). LE. UNSORT(I+GAP)) GO TO 40 
A = UNSORT(I) 
UNSORT(I) = UNSORT( 1+GAP) 
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UNSORT(I+GAP) = 
COUNT = COUNT + 
40 CONTINUE 
IF (COUNT. GT. O) GO 7e ze 
IF (GAP, GT. 1) GO 707: 
RETURN 
END 


A 
1 


Vedede dere dede edele eke ee de ede eed dee ede ee ee ede dek ke Ik eek Vee Ve ee Ie Ee eek Fees eeste 


SUBROUTINE NORMAL( INPUT, RESULT) 


desse ee de dede see dedere deve ve ve ve ve ve eve eve de ee vede ole ole ole ole ole tente ste ار از سر رہ‎ EE EI EE IE EE ee ee der 


* Subroutine to compute probabilities for the standard normal * 


* distribution. * 
Ee ME EE IE EE EI MIE IE ME EE MIE DES EE SE SESSE EE IE EIE EE EE EISE EIE Ie 


* Declare variables 


LOGICAL NEG 

DOUBLE PRECISION AA, BB, CC, DD, EE, FF, Z, X 

REAL RESULT, INPUT 

DATA AA/0.049867347D0/, BB/0.0211410061D0/, ርር/0. 003277626300/ , 
CDD/0. 0000380036D0/, EE/0.0000488906D0/, FF/0.000005383D0/ 


* Prepare input 


NEG = . FALSE. 

IFC INPUT. bio NEG = 2 ٣ 
Z = DBLEC INPUT) 
BEONE GE ON RIE T 


EA 
e 


Apply formula 


130 ^ AA*Z * BB*Z**2DO 4 CC*Z**3DO 4 DD*Z**4DO 
X 4 EE*Z**5DO0 + FF*Z**6D0 

X***( - 16D0) 

130 - 0.5DO*X 


DS PS PS PS 
١ 


Prepare output 


RESULT = REAL(X) 

IF(NEG. EQV. . TRUE. ) RESULT = 1 - RESULT 
RETURN 

END 


e Jedese vede Je Jedede Jede Jede JeJesye Je ee Je Je Jede Je Je Je Je Jede Je fe Je fe ሃሮን Je Jede Je dye dede Jede de Je Je e de Je dede Je Jede Jede Jede stede dese‏ حا لی کیہ مکی مکی لی پل 


SUBROUTINE INVNOR( INPUT , RESULT) 
ee ee de Dee Tee Vee DEE ETER Tee PIT RR Hekke ee ee ee ee Te DERE RITS Fe esse 


* Subroutine to compute quantiles of the standard normal distribution. * 


Iede seed Ve Tee Tee ede ee Te Tee Te de ede Vee Te Ve Tee Le Te Pe Pe TT Tee Te Ve TER Vee ee ede ek Vee ee Te Te Te de Re Te ees 
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* Declare variables 


DOUBLE PRECISION A, B, C, D, E, F, P, T, YY 

REAL INPUT, RESULT 

LOGICAL LESS 

DATA A/2.515517D0/, B/0.802853D0/, C/0.010328D0/, D/1. 432788D0/, 
CE/0.189269D0/, F/0.001308D0/ 


* Prepare input 
LESS = . FALSE. 
P = DBLE( INPUT) 
ትርክ Hi ao 5) LESS = . TRUE. 
WEOLESSTEOMSTEALSES JP = 1DO - P 
* Apply formula 


T = DSQRT(DLOG( 1D0/P**2D0) ) 


SSS TET E GTT**290 
YY ST 190 * DAT EXT**2D0 + F*T**3D0>) 
YY =T - YY 


* Prepare result 


ከ ከ፡ ንቲ EOV.. TRUB. ) YY = -YY 
RESULT = REAL( YY) 

RETURN 

END 


45 


APPENDIX E. SIMTBED DRIVER FOR BOOTSTRAP 


The program listed here is an example for the drivers used in the simulations under 
SIMTBED [Ref. 9. SIMTBED is written in FORTRAN and operates under the IBM 
Professional FORTRAN, which is a. prerequisite for any application. The drivers, 1. e. 
the user input has to be written in this programming language. 

The main part of the driver is the general call to SIMTBED with all the parameters 
like sample size, number of replications, destination and form of the output etc.. This 
part basically follows the instructions in the manual. 

The subroutines constitute the part of the driver, which is specific for each problem. 
Here the user has to set up the specific simulation by choosing the distribution of the 
random variates and by programming how the statistical estimates are to be computed. 
The subroutines LIREI and LIRE? im the driver are written to compute the bootstrap 
estimate of the y-intercept respectively the slope in normal linear regression. In all sim- 


ulations involving normal linear regression the same basic setup was used. 
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Te TE EE KIEKIE EE EIE EIE EIE EE FEE EIE EIE IE TEE VEE ESE EE KERR KEER RR RE EE KRIEK EE deese 


* SIMTBED driver for normal linear regression es 
Main program 2 


EE EER dde 


als 
4% 


eese jede ves ede EIE ey eee eee se 


* Declare variables 


REAL*4  Y(3500),YMIN, YMAX 

CHARACTER*120 T1,T2 

REAL*8 IX1,IX2,IX3,IX4,IX5, IX 

INTEGER  N,M,NE(8) , L, D,RG,SEI,SVS,NEST,NCOLRNDX( 3) , IFILE,IBWPRT, 


C MSE, 


NEREDE, IRSTR 


REAL VMSE(8, 555 VMX1(8, 4),VMX2(8,4),VMX3(8,4) , VMX4(8,4) , VMX5(8,4) 


EXTERNAL LIRE1, 


* Input of 


DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 
DATA 


LIRE2 


SIMTBED parameters 


N/3000/ 

M/ 1/ 

NE/ 10,20,30,40,50,50,50,50/ 
8 

D/ 0/ 

RG/ 0/ 

SEI/ 0/ 

SVS/ 0/ 

YMIN/ 0./ 
YMAX/ 0. / 

IX/ 88771.D0/ 
IFILE /0/ 
NPRT /1/ 

MSE X /0/ 
VMSE /40*0/ 
IPR  /0/ 
IBIV /0/ 
IRSTR /1/ 


DATA ICOLOR/O/, IBWPRT/1/, NCPRT/1/, NCOLRNDX/1,2,7/ 


* Set output parameters 


DATA T1/'Y-INTERSECT NORMAL LINEAR REGRESSION 
C (100 Bootstrap replications)'/ 

DATA T2/'SLOPE NORMAL LINEAR REGRESSION 

C (100 Bootstrap replications)'/ 


OPEN(06,FILE-'LIRE100. OUT' ,ERR2999, IOSTAT-IER) 

OPEN( 05 ,FILE='CON' ,ERR=999 , IOSTAT=IER) 

OPEN(02,FILE=' LIRE100. RST' ,ERR=999 , IOSTAT=IER , FORM=' UNFORMATTED’ , 
C ACCESS='SEQUENTIAL' ) 

OPEN(01,FILE=' LIRE100. DAT’ , ERR=999 , IOSTAT=IER , FORM=' FORMATTED’ , 
C ACCESS=' SEQUENTIAL’ ) 


* Generator parameters 


NEST=2 
NSR=10 
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IX1=1X 
IX2=IX 
IX3-IX 
IX4-IX 
IX5-IX 


* Make the call to SIMTBED 


CALL SMTBED(IX1,IX2,IX3,IX4,IX5, Y, N,M,NE,L,D, NSR,RG,SEI,SVS, 
C YMIN,YMAX,NEST,LIRE1,T1,LIRE2,T2,LIRE1,T1,LIRE1,T1,LIRE1,T1, 
C IFILE,NPRT,MES,VMSE, IPR,VMX1,VMX2,VMX3,VMX4 ,VMX5 , IBIV, IRSTR, 
C ICOLOR,IBWPRT,NCPRT,NCOLRNDX) 


STOP 


999 CONTINUE 
WRITE(6,*) 'ERROR OPENING FILE 1, 2 OR 6' 
END 


مل مک مل مل مل مک مل پک پگ مل مب مل جک وب مب مک مک مک مل بک مک مک" مک بل مک مک مک مل مک مک مل مب Je desee dese eese dede Jevese Je vede ye vede Jede dede lee delete‏ مک مل م'و ہپ مب مب بب بژذ جب Veste‏ 


SUBROUTINE LIRE1( ISEED,N,EVAL) 


see IE IE EIE y eyes ev ede سیا مر سا‎ EIER ار ہا سے ارس از‎ EIE IE EI EIE EIE IE SE ESE ESE IE EIE IE IE IE EE EE le leste le je le lede te sle teste este stee Ve 


* Subroutine to evaluate the first estimator, the Y-intercept 2 
SEER SEE EE EE SESSE IE ንጅንኛንሮንኛሣኝሮንሮንኞሮንኞኝሮንሮኝሮኝኛሪኝ።ኝ5ንኝኛንዩንሮንኛንኛንሮንኞንኝሮኝሮኝሮኝዮንኛንሮኝሮኝሮንኞንኞሮንኞንኛንኝኛኝሮንሮንሮንሮኝ፣ንርንኛኝሮንሮኞኝሮንሮንኛኝኖኝኛንኝኛኝኝኞኝኞኛ 


* Declare variables 


INTEGER N, BREP, DRAW, NN 

REAL BETAO, BETA1, BO, B1, BHATO, BHAT1, VAR, XBAR, YBAR, 
C NUM1, DENOM, EVAL 

REAL*8 ISEED 

PARAMETER (VAR=0.5, BETAO=1.5, BETA1=0.8, NN=50) 

REAL X(NN), Y(NN), EPSI(NN), RAND(NN), NORRAN(NN) 


BREP = 100 
* Compute the x-values and related results 


DO 99, I= 1, N 
X(I) = REAL(I)*10. /REAL(N) 
99 CONTINUE 


XBAR = 0 
Do dee = 
XBAR = 
I CONTINUE 


1, N 
XBAR + X(1)/N 


DENOM = 0 
00 2, ፲ = 1, ከ 
DENOM = DENOM + (X(I) - XBAR)**2 
2 CONTINUE 
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* Create original pairs of observations and compute parameters 


CALL LNORPC(ISEED,NORRAN,N) 


YBAR = 0 

DO 10, I= 1, እከ 
Y(I) = BETAO + BETA1*X(I) + SQRT(VAR)*NORRAN(I) 
YBAR = YBAR + Y(I)/N 


10 CONTINUE 


NUM1 = 0 
DO 11, I = 1, N 
NUM1 = NUM1 + (X(I) - XBAR)*(Y(I) - YBAR) 
11 CONTINUE 


Bl 
BO 


NUM1/DENOM 
YBAR ~ B1*XBAR 


* Compute the epsilons 


DO 12, I= 
EPS Ed) 
12 CONTINUE 


1, N 
- Y(I) - BO - B1*X(I) 


* Do the bootstraps 


SEX] Koc 1. 


DRAW = INT(N*RAND(K)) + 1 
Y(K) = BO + BI*X(K) + EPSI(DRAW) 
YBAR = YBAR + Y(K)/REAL(N) 
21 CONTINUE 


DO 22, K= 1, N 
NUM1 = NUM1 + (Y(K) - YBAR)*(X(K) - XBAR) 
22 CONTINUE 
B1HAT = NUM1/DENOM 
BOBAR = BOBAR + (YBAR - Bl1HAT*XBAR)/REAL(BREP ) 
20 CONTINUE 
EVAL = BOBAR 


RETURN 
END 


ee de eke ese de Jede ye eee TEE FR ER EK ede Je lede dee tee ee Veere de Je Jede Jede Ve eee Je Ae Jede Je ee edes 
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SUBROUTINE LIRE2(ISEED,N,EVAL) 


Seed Tee ee dees Ee Ve Vee ee Me DE ee RR ER TR IERE PIET 


Subroutine to evaluate the second estimator, the Slope ዣ 


> መ. መ af. -> = ራሙ + ele -> ef, > ele > -መ መ - ap መ > መ መ m Ta Ge ap Ma) ab Hd ap he! ap Ge ah a Had da Ma) ap de ap Oe! ap ے۹ ا 2 ل‎ > ቃታ መ > e's ( ለ - -> መቃ 
Jededededesedededededededeveyedesededevevedededededededededesedededevyeve Jedesededede Je dedede Jede Je dede ye dede dededede dese Jeve ye tele re dere Te leder 


m 
er 


* Declare variables 


INTEGER N, BREP, DRAW, NN 

REAL BETAO, BETA1, BO, B1, BHATO, BHAT1, VAR, XBAR, YBAR, 
C NUM1, DENOM, EVAL 

REAL*8 ISEED 

PARAMETER (VAR=0.5, BETAO=1. 5, BETA1=0.8, NN=50) 

REAL X(NN), Y(NN), EPSI(NN), RAND(NN), NORRAN(NN) 


BREP = 100 
ና Compute the x-values and related results 


DO 99, I= 1, N 
X(I) = REAL(I)*10. /REAL(N) 
99 CONTINUE 


XBAR = 0 

Du 

XBAR 
1 CONTINUE 


1, N 
XBAR + X(I)/N 


, N 
DENOM = DENOM + (X(I) - XBAR)**2 
2 CONTINUE 


* Create original pairs of observations and compute parameters 


CALL LNORPC( ISEED ,NORRAN,N) 


YBAR = 0 

DO 10, I =1, N 
Y(I) = BETAO + BETA1*X(1) + SQRT(VAR)*NORRAN( I) 
YBAR = YBAR + Y(I)/N 


10 CONTINUE 


NUM1 = 0 
DO 11, I= 1, N 
NUM1 = NUM1 + (X(I) - XBAR)*(Y(I) - YBAR) 
11 CONTINUE 


Bl 
BO 


NUM1/DENOM 
YBAR - B1*XBAR 


* Compute the epsilons 


DO 12, 1 = 1, እከ 
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EPSKI) S YU) = BO BI*X(I) 
12 CONTINUE 


* Do the bootstraps 


BOBAR = 0 
BIBAR = 0 
Dog eJ — I. BREP 

CALL LRNDPC(ISEED,RAND,N) 

YBAR = 0 

NUM1 = 0 

000 S= N 


DRAW = INT(N*RAND(K)) + 1 
Y(K) = BO + B1*X(K) + EPSI(DRAW) 
YBAR = YBAR + Y(K)/REAL(N) 

21 CONTINUE 


DO 22 IK E1, N 
NUM1 = NUM1 + (Y(K) - YBAR)*(X(K) - XBAR) 
22 CONTINUE 


B1HAT = NUM1/DENOM 


B1BAR = BIBAR + B1HAT/REAL(BREP) 
20 CONTINUE 
EVAL = BIBAR 
RETURN 
END 
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